Exact Statistical Characterization of 2×22\times2 Gram Matrices with Arbitrary Variance Profile

This paper is concerned with the statistical properties of the Gram matrix $\mathbf{W}=\mathbf{H}\mathbf{H}^\dagger$, where $\mathbf{H}$ is a $2\times2$ complex central Gaussian matrix whose elements have arbitrary variances. With such arbitrary variance profile, this random matrix model fundamentally departs from classical Wishart models and presents a significant challenge as the classical analytical toolbox no longer directly applies. We derive new exact expressions for the distribution of $\mathbf{W}$ and that of its eigenvalues by means of an explicit parameterization of the group of unitary matrices. Our results yield remarkably simple expressions, which are further leveraged to study the outage data rate of a dual-antenna communication system under different variance profiles.Comment: 6 pages, 1 figure, 1 tabl

Scalable Online Betweenness Centrality in Evolving Graphs

Betweenness centrality is a classic measure that quantifies the importance of a graph element (vertex or edge) according to the fraction of shortest paths passing through it. This measure is notoriously expensive to compute, and the best known algorithm runs in O(nm) time. The problems of efficiency and scalability are exacerbated in a dynamic setting, where the input is an evolving graph seen edge by edge, and the goal is to keep the betweenness centrality up to date. In this paper we propose the first truly scalable algorithm for online computation of betweenness centrality of both vertices and edges in an evolving graph where new edges are added and existing edges are removed. Our algorithm is carefully engineered with out-of-core techniques and tailored for modern parallel stream processing engines that run on clusters of shared-nothing commodity hardware. Hence, it is amenable to real-world deployment. We experiment on graphs that are two orders of magnitude larger than previous studies. Our method is able to keep the betweenness centrality measures up to date online, i.e., the time to update the measures is smaller than the inter-arrival time between two consecutive updates.Comment: 15 pages, 9 Figures, accepted for publication in IEEE Transactions on Knowledge and Data Engineerin

When Two Choices Are not Enough: Balancing at Scale in Distributed Stream Processing

Carefully balancing load in distributed stream processing systems has a fundamental impact on execution latency and throughput. Load balancing is challenging because real-world workloads are skewed: some tuples in the stream are associated to keys which are significantly more frequent than others. Skew is remarkably more problematic in large deployments: more workers implies fewer keys per worker, so it becomes harder to "average out" the cost of hot keys with cold keys. We propose a novel load balancing technique that uses a heaving hitter algorithm to efficiently identify the hottest keys in the stream. These hot keys are assigned to $d \geq 2$ choices to ensure a balanced load, where $d$ is tuned automatically to minimize the memory and computation cost of operator replication. The technique works online and does not require the use of routing tables. Our extensive evaluation shows that our technique can balance real-world workloads on large deployments, and improve throughput and latency by $\mathbf{150\%}$ and $\mathbf{60\%}$ respectively over the previous state-of-the-art when deployed on Apache Storm.Comment: 12 pages, 14 Figures, this paper is accepted and will be published at ICDE 201

Lunar impact flashes from Geminids, analysis of luminous efficiencies and the flux of large meteoroids on Earth

We analyze lunar impact flashes recorded by our team during runs in December 2007, 2011, 2013 and 2014. In total, 12 impact flashes with magnitudes ranging between 7.1 and 9.3 in V band were identified. From these, 9 events could be linked to the Geminid stream. Using these observations the ratio of luminous energy emitted in the flashes with respect to the kinetic energy of the impactors for meteoroids of the Geminid stream is estimated. By making use of the known Geminids meteoroid flux on Earth we found this ratio to be 2.1x10$^{-3}$ on average. We compare this luminous efficiency with other estimations derived in the past for other meteoroid streams and also compare it with other estimations that we present here for the first time by making use of crater diameter measurements. We think that the luminous efficiency has to be revised downward, not upward, at least for sporadic impacts. This implies an increase in the influx of kilogram-sized and larger bodies on Earth that has been derived thus far through the lunar impact flash monitoring technique